Speculative Query Processing

Speculation is an every day phenomenon whereby one acts in anticipation of particular conditions that are likely to hold in the future. Computer science research has seen many successfull applications of speculation: modern processors, for example, speculate on the run-time properties of a program and decide to pre-execute instructions accordingly. We draw inspiration from these techniques and introduce speculation to query processing. Our approach is based on a visual query interface that monitors the construction of a query and takes advantage of the user ‘think time’. In particular, based on the features of the partial query specified at any point, the interface prepares the database by issuing asynchronous manipulations to it that are likely to make the final query (or even queries further into the future) more efficient. Furthermore, the interface applies machine learning techniques on past user actions and builds a user-behavior model that guides speculation and deals with future uncertainty. We formalize speculative query processing as an optimization problem and derive algebraic properties of the corresponding cost model that are sufficient to address the complexities of the particular optimization. We have implemented our framework on top of an existing commercial database system and have evaluated its effectiveness experimentally, with actual user traces. Our results show that speculation outperforms normal query processing, reducing query execution time by an average of 35% and achieving performance improvements of more than 90% on certain queries. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee and/or special permission from the Endowment. Proceedings of the 2003 CIDR Conference